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FINAL  PROGRESS  REPORT 


STATEMENT  OF  THE  PROBLEM  STUDIED  AND  SUMMARY  OF 
THE  MOST  IMPORTANT  RESULTS 

In  article  [1]  we  study  the  topic  of  Backward  Stochastic  Differential  Equa¬ 
tions  (BSDEs)  and  establish  existence  and  uniqueness  results  for  solutions  to 
such  equations  in  the  presence  of  two  reflecting  barriers,  generalizing  previ¬ 
ous  works  on  BSDEs,  in  particular  El  Karoui  et  al.  (1997).  We  also  demon¬ 
strate  that  the  solution  coincides  with  the  value  of  a  certain  Dynkin  game,  a 
stochastic  game  of  optimal  stopping.  The  game  involves  two  players,  each  of 
whom  can  decide  to  stop  it  at  a  random  time  of  his  choice;  upon  termination 
a  certain  amount  is  paid  by  one  of  the  players  to  the  other.  Moreover,  the 
connection  with  the  BSDE  enables  us  to  provide  a  deterministic  approach  to 
the  game.  One  can  also  think  of  the  solution  as  a  “guided  missile”  that  has 
to  hit  a  target  at  some  future  time  T,  but  is  also  required  to  remain  inside  a 
certain  (moving  and  random)  region  before  that  time. 

The  discrete-time  version  of  these  results,  including  both  the  constrained 
Dynamic  Programming  Equation  (analogue  of  the  BSDE)  and  the  pathwise 
approach  to  the  discrete-time  stochastic  game,  are  developed  in  paper  [2]. 

Article  [3]  studies  the  Monotone  Follower  Problem  in  Singular  Stochastic 
Control  and  its  connections  with  Optimal  Stopping.  These  connections  are 
developed  in  the  context  of  the  “irreversible  investment”  problem  in  math¬ 
ematical  economics,  first  considered  by  K.  Arrow,  and  allow  “elementary” 
treatment  of  this  problem  in  complete  generality. 

In  article  [4]  we  completed  our  definitive  treatment  of  the  general  multi¬ 
armed  bandit  (dynamic  allocation)  problem  in  continuous  time,  for  arms 
that  are  not  necessarily  independent  or  Markovian.  This  work  complements 
our  previous  major  contribution  to  this  subject,  which  provides  a  similar 
treatment  for  the  discrete-time  case  (El  Karoui  h  Karatzas  (1993)).  Notions 
and  results  from  time-changes,  optimal  stopping  theory,  and  multi-parameter 
martingale  theory  were  used  extensively  in  this  treatment,  and  the  tradi¬ 
tional  independence  assumption  was  replaced  by  the  famous  condition  (F.4) 
of  Cairoli  &  Walsh,  which  allows  for  some  dependence  in  the  evolution  of  the 
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various  arms  (or  projects).  In  this  work  we  also  introduced  a  ’’synchronisa¬ 
tion  identity”  for  allocation  strategies,  and  brought  out  its  central  role  in  the 
context  of  dynamic  allocation:  this  condition  is  both  necessary  and  sufficient 
for  optimality  in  the  case  of  decreasing  rewards,  and  leads  to  an  allocation 
strategy  with  all  the  important  properties  in  the  general  case  as  well,  namely 

(i)  optimality  in  the  dynamic  allocation  problem, 

(ii)  optimality  in  a  dual  (minimization)  problem,  and 

(iii)  the  ’’index-type”  property  of  Gittins. 

Even  when  specialized  to  the  case  of ’’diffusion”  or  ’’Brownian”  Bandits, 
this  synchronisation  propery  leads  to  deep  results,  as  we  demonstrate  in  the 
paper  by  examples.  We  believe  that  this  is  a  seminal  study  of  the  problem, 
and  that  it  will  lead  to  renewed  interest  in  this  and  similar  questions. 

In  work  [5],  we  solve  the  following  adaptive  control  problem:  to  maximize 
the  probability  P[X(T)  —  1],  subject  to  the  dynamics  dX(t)  =  u{t)[Bdt  + 
dW(t)],X(0)  =  x,  0  <  x  <  1,  where 

(i)  B  is  a  random  variable  with  known  distribution,  independent  of  the 
Brownian  motion  W (•), 

(ii)  u(-)  is  a  real- valued  control  process,  square-integrable  over  the  interval 
[0,  T]  with  probability  one,  and  adapted  to  the  filtration  generated  by  the 
Brownian-motion- with-unknown-drift  process  Y  (t)  —  W  (t)  +  Bt,  and 

(iii)  the  state-process  X(-)  takes  values  in  the  interval  [0,1]  and  is  ab¬ 
sorbed  when  it  reaches  either  one  of  the  endpoints  of  the  interval. 

We  obtain  very  explicit  expressions  for  both  the  optimal  control,  and 
for  the  value-function  of  this  Bayesian  adaptive  control  problem.  In  the 
process,  we  also  obtain  explicit  solutions  to  the  parabolic  Monge-Ampere- 
type  equation 


2  QyyQs  =  Q  xxQyy  ( Qxy ) 


which  is  related  to  the  HJB  equation  of  this  problem. 

In  article  [6]  we  develop  a  general  theory  of  Backward  Stochastic  Differ¬ 
ential  Equations  with  convex  constraints  on  the  gains  (or  intensity-of-noise) 
process.  We  establish  existence  and  uniqueness  of  a  minimal  solution  under 
Lipschitz  and  convexity  assumptions  on  the  drift  coefficient.  We  also  show 
that  the  minimal  solution  can  be  characterized  as  the  unique  solution  of  a 
functional  stochastic-control-type  equation.  This  representation  is  impor¬ 
tant  because  it  provides  an  explicit  scheme  -  a  penalization  method  -  for 
constructing  the  minimal  solution.  Moreover,  we  provide  a  similar  analysis 
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in  the  more  general  case  in  which  there  is  also  a  reflecting  lower-barrier  for 
the  state-process. 

The  paper  [7]  discusses  the  problem  of  optimal  stopping  for  a  controlled 
diffusion  process  on  the  unit-interval  I  =  [0, 1]  with  absorbing  barriers,  and 
with  drift/diffusion  pair  that  can  be  selected  at  any  instant  t  G  [0,oo)  from 
a  set  depending  only  on  the  current  position.  We  reduce  this  mixed  optimal 
stopping/control  problem  effectively  to  two  optimal  stopping  problems,  and 
describe  explicitly  its  optimal  policies. 

In  technical  report  [8]  we  generalize  our  previous  results  [1]  on  exis¬ 
tence  and  uniqueness  of  solutions  to  one-dimensional  BSDEs  to  the  multi¬ 
dimensional  and  Forward-Backward  case,  in  which  the  “backward”  process 
has  to  hit  the  “forward”  process  at  time  T,  and  they  are  both  restricted  to  re¬ 
main  in  a  certain  region.  In  the  case  of  a  Markovian,  one-dimensional  model, 
we  characterize  the  solution  of  the  Forward-Backward  SDE  as  the  unique 
viscosity  solution  to  a  quasilinear  variational  inequality  (obstacle  problem) 
with  a  Neumann  boundary  condition. 
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